feat: 增加事务化文件系统复制能力 - #4185
Open
KCHENPENGFEI wants to merge 20 commits into
Open
Conversation
KCHENPENGFEI
force-pushed
the
codex/feature-ov-cp
branch
from
August 21, 2026 06:19
59df0cc to
57842fc
Compare
Collaborator
|
@KCHENPENGFEI 可以直接在 ragfs 的 filesystem.rs 中实现 原子化 cp 呢? |
Collaborator
|
Thanks for the substantial work on transactional copy! This branch currently has merge conflicts with main ( |
KCHENPENGFEI
force-pushed
the
codex/feature-ov-cp
branch
from
August 24, 2026 06:55
57842fc to
8c9bbba
Compare
baojun-zhang
requested changes
Aug 24, 2026
Collaborator
There was a problem hiding this comment.
这个 PR 的方向是给 VikingFS / API / CLI 增加事务化 cp,并在 cp/mv 后迁移或刷新语义数据。当前实现还有几处会导致数据损坏、锁保护失效或取消后绕过补偿的问题,需要先修完再合并。
典型场景:S3 后端复制 20 MiB 文件时,RAGFS 进入 raw copy 分块写目标,但 S3 write 是整对象 put,offset 不生效,最终对象只剩最后一块。另一个场景是目录 cp 发布目标后只持有目标 Exact 锁,向量迁移失败回滚时可能把其他请求并发写入目标子目录的数据一起删除。
结论:REQUEST_CHANGES。
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
force-pushed
the
codex/feature-ov-cp
branch
from
August 24, 2026 13:38
3f3124f to
8addac9
Compare
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 24, 2026
baojun-zhang
requested changes
Aug 24, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
pushed a commit
to KCHENPENGFEI/OpenViking
that referenced
this pull request
Aug 25, 2026
KCHENPENGFEI
force-pushed
the
codex/feature-ov-cp
branch
2 times, most recently
from
August 25, 2026 14:48
d8b02a9 to
c0ce194
Compare
added 20 commits
August 31, 2026 22:19
KCHENPENGFEI
force-pushed
the
codex/feature-ov-cp
branch
from
August 31, 2026 14:35
c0ce194 to
ecb4a30
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更说明
新增文件与目录复制能力,覆盖 VikingFS、HTTP API 和
ovCLI。复制时同步迁移已有向量记录,避免重复解析、文件级 VLM 和 embedding;提交后异步重建受影响父目录的语义信息。人工参与
关联事项
接口提案已经在项目 Discussion 中批准。
变更类型
主要改动
cp能力新增
POST /api/v1/fs/cp和ov cp:ov cp支持单文件复制,ov cp -r支持目录递归复制。RAGFS 原子发布与失败补偿
AGFS 复制已下沉到 RAGFS:
mv继续采用“复制目标 → 迁移向量 → 删除源”的补偿式流程。这里不是跨 AGFS 与 VectorDB 的强一致两阶段提交:AGFS 内部通过 staging 原子发布,跨存储一致性通过严格校验和补偿回滚保证。
父目录语义刷新
cp完成后异步刷新目标父目录;mv完成后异步刷新源、目标父目录。刷新消息使用generation_trigger="content_copy":接口只等待消息入队,不等待语义处理完成。
文档
补充中英文 HTTP 接口、端点总览和 CLI 使用文档。
局限性
当前没有提供 AGFS 与 VectorDB 的强一致快照。如果
add-resource已完成 AGFS 落盘,但向量仍在生成,cp可能只复制当时已经写入的向量,导致目标向量不完整。正常情况下,
add-resource在语义和 embedding 阶段持有目录 Tree Lock,重叠的cp通常会返回409 path_busy,实际竞态窗口较短,本期暂不增加索引完成屏障或增量补齐机制。父目录语义刷新是异步的,
cp/mv返回成功时,新的 overview 和 abstract 可能尚未生成。测试
本次 rebase 后的定向验证:
ov_cli的cp请求测试通过。cargo check -p ragfs-python通过。ov cp命令。全量 filesystem router 测试仍有一个
upstream/main已存在的attrsFilter DSL 用例失败,与本 PR 无关;本机ragfs-python单元测试链接阶段缺少 Python 符号,但cargo check已通过。检查清单